Event-Driven Architecture Protocols

Motivation#

The HTTP protocol is suitable for transferring data between client and server. However, for real-time communication, such as instant messaging, live streaming, gaming, and so on, HTTP falls short due to the connection terminating after a response from the server. The HTTP-based techniques to fetch updated data also have limitations. For example, short polling sends too many unnecessary requests, long polling uses two separate connections for sending and receiving data, and HTTP streaming allows only half-duplex communication. Event-driven architecture addresses all these limitations of HTTP-based techniques.

Event-driven architecture#

Event-driven architecture (EDA) is an approach that centers around events for data and applications. The event-driven architecture enables applications to act or react to those events. A client gets the data whenever an event is triggered in the application without requesting the server. The problems associated with short and long polling are eliminated by using an event-driven approach. This is very much in contrast with the traditional REST API approach to request the updated data after a certain interval or when needed. The illustration below explains the concept.

Note: The event-driven protocols require the client to subscribe to a particular topic for receiving event updates, often specifying the endpoint (a callBack URL) to post the updates. The callBack method via link allows connecting directly with the host even if it is not accessible via IP address. This URL is unique for each subscription. The pub-sub uses functionality of capability URLs to define callback URLs.

In these cases, the client wants to be informed that something has happened. In this case, there is no request or asking paradigm (which explains that the client wants to be informed about the events without sending a request (request/asking) to the server); therefore, no response is expected. An example of this can be a user posting on Instagram. The followers get notified about the post, the reactions to the post, and the posted comments, along with the event payload with each notification. Event payload or message payload is some extra information sent alongside an event notification. For example, a user's information, reaction type and count, comment messages, and so on.

The event-driven architecture provides an asynchronous environment. For example, when an event triggers, the system performs the intended action immediately and doesn’t wait for the synchronous delivery of other events. In other words, it doesn’t wait for a response from event listeners. This is an ideal architecture to use when clients need to be notified about the changes occurring in the backend.

Types of event-driven protocols#

This lesson will discuss the primary event-driven protocols that are used to push data from the server. Furthermore, we’ll elaborate on how APIs based on event-driven architecture perform well in different real-time scenarios. The following is a list of event-driven protocols we’ll discuss in this lesson:

• WebHooks

• Server-Sent Events (SSE)

• WebSub

• WebSocket

Webhooks#

Webhook is a one-way event-driven communication protocol based on HTTP. It enables applications to send real-time information when a particular event gets triggered. In a webhook, the server (consumer) must first subscribe to a particular topic through a specified HTTP/HTTPS endpoint with another application's server (producer). After the subscription, any new events are automatically notified by the server. The webhook protocol is used when two servers or applications have to communicate with each other. Let’s consider the example of the Slack application where a user wants to receive a notification for meetings scheduled on Google Calendar. If this communication were to take place using the request-response architecture, it would look something like the diagram depicted below:

In the illustration above, Slack periodically requests Google Calendar for an update using the request-response architecture. As shown in the diagram above, Google Calendar didn’t have an update for the first couple of requests, but provided an update to Slack on the third request. For organizations, there can be hundreds of customers and dozens of meetings or other scheduled events. If Slack starts polling each user from the Google Calendar application, it becomes wasteful for both parties involved. Therefore, using the request-response architecture is not ideal for these kinds of scenarios. This situation requires the use of event-driven architecture. Let's see how we can improve this experience using webhooks. Take a look at the illustration below:

In the illustration above, the Slack application first subscribes to a particular event with the Google Calendar application. After a successful subscription, Google Calendar notifies Slack when there is an update for the subscribed event. Therefore, we can see how webhooks reduce the overall communication required for event updates between two applications. Another advantage of this approach is to avoid reaching the application's rate limit.

Despite the advantages, webhooks also have the following drawbacks:

• Not every application supports webhook integrations.

• It isn’t suitable in situations where an application wants to know only the final result instead of an update of each event.

• It only pushes the events to the consumers; if a server cannot receive them, there is no mechanism to explicitly request that event from the server.

• A consumer doesn’t know when the server is down or if it fails to send events. This can result in an assumption that the events are not occurring at all.

Applications that provide different services to other applications through the webhook architecture include:

• Google uses webhooks for Google Assistant and Google Calendar services.

• Meta uses webhooks for payments, messengers, and social graph services.

Other webhook applications are Twilio, Slack, Shopify, Stripe, and many more.

Server-Sent Events (SSE)#

Server-Sent Events (SSE) are a unidirectional (server-to-client) server push approach and a component of the HTML5 specification. Here, a server push means that the specified client automatically gets data from the server after the initial connection has been set up. In SSE, the client requests the URL through a standard EventSource interface to enable an event stream from the server. Additionally, the client defines the text/event-stream under the Accept header in the request. Here, the event-stream represents a media type known as Multipurpose Internet Mail Extensions (MIME). Before initiating the request, the client-side application also checks the server-sent event support on the client (browser).

This approach is mainly used for the notification system, where clients get notified by the server upon update. Let's see the illustration below to understand the SSE approach:

In the illustration above, the client automatically gets notifications until the client or server terminates the connection. Both the SSE and webhook protocols are similar in functionality; the only difference is that webhooks send events to servers, and SSE sends events to clients.

The drawbacks that come with the SSE approach are as follows:

• When we use SSE over HTTP/1.1, the browsers allow a limited number of connections per domain because each requested resource requires a separate TCP connection. However, HTTP/2 has resolved this issue by multiplexing all requests and responses over a single TCP connection, but most applications still use HTTP/1.1.

• Another major problem with SSE is that it only allows text-based streaming due to data format limitations, such as not supporting binary data. So, we can’t use it for other streaming data such as video, audio, and so on.

Applications use SSE for different purposes. For example, Twitter, Instagram, and Meta use SSE for newsfeed updates. Stock applications use SSE for real-time stock updates. Sports applications use SSE for live score updates.

WebSub (Pub-Sub)#

WebSub is another event-driven architecture for distributed publish-subscribe communication, also known as pub-sub, and PuSH. WebSub has three main components, which is one of the reasons why it is unique compared to other event-driven architectures. 

• Publisher: The content producer publishes the specified topic event information to the defined Hubs.

• Hub: A middleman that decouples publishers and subscribers. It serves as a content distributor who takes the specified content from the publisher and pushes it toward the subscriber.

• Subscriber: A consumer who receives the update for particular content (known as a topic) from the subscribed Hubs publicized by the publisher. Whenever a publisher produces new content related to a specific topic, the Hub pushes them to the subscriber(s).

Let's see the illustration below to understand how the above three components work together.

In the illustration above, the subscriber first asks for the defined Hubs from a publisher. After getting the Hub details, subscribers send the subscription request for a specific topic in the resource URL to a particular Hub. The Hub uses this topic resource URL to send updates delivered by the publisher. The structure of the subscription/unsubscription request is defined below:

• hub.mode: A string containing subscribe or unsubscribe to define the request.

• hub.topic: The resource URL of the topic to which the subscriber wants to subscribe or unsubscribe.

• hub.callback: A callback URL where the Hub responds with subscription details.

Note: The callback URL allows connecting directly with the host, even if it isn’t accessible via IP address (it isn’t dependent on the IP address of the host). This URL is unique for each subscription.

The Hub verifies the subscription request by sending a separate GET request to the subscriber. After that, the subscriber responds with a 200 status code to confirm the subscription. Upon successful subscription, the Hub sends notifications to all subscribers for the specific topic when it receives updates from the defined publisher.

WebSub is ideal when multiple clients require updates from the same producers. Another unique thing that comes with WebSub is the built-in security feature known as verification of intent. As shown in the illustration above, verification of intent is a kind of handshake that takes place before receiving content in the subscription request from the subscriber. The intent is verified by the Hub as an extra security layer to avoid subscription requests from attackers posing.

Note: The WebSub network stops working when the Hub goes down.

WebSocket#

WebSocket is an event-driven protocol that enables bidirectional communication between the client and servers. It differs from the protocols mentioned above because they target sending events only from the server side (unidirectional). Whereas in WebSockets protocols, communicating parties can exchange data whenever needed and without any restrictions. More details on WebSocket were discussed in this course’s lesson on WebSockets.

Quiz#

Match the information in the two columns in the widget below by selecting the correct option.

Summary#

The event-driven architecture protocols provide users with the intended data without making a request to the server whenever an event triggers. We learned that event-driven architecture is a suitable option to address the limitations of HTTP techniques for real-time communication. Additionally, the details of each event-driven architecture protocol enable us to select the most ideal protocol to use according to the required communication. The following table depicts the key features of each of the protocols mentioned above:

For example, SSE is suitable if an application wants to send events to clients with textual payloads only. Similarly, if targeted consumers are clients and servers, and there is no limitation of data formats, then the WebSub protocol is more ideal.